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Tests (PPST) is described. Panels of experts from teacher education 
institutions and local school districts performed the test item 
review, focusing on test validity and minimum requirements for 
admission to teacher education programs. Participants (136 teacher 
educators, 108 high school teachers and curriculum specialists, and 
15 professional educators) were sent preview information before 
participating in the review. The teacher educator panel reviewed 
every test item; (1) for its relevance to successful performance in 
their institution's teacher preparation program, and (2) to determine 
whether or not a student should be required t correctly answer the 
item in order to be admitted to a teacher education program. 
Judgments were also made on the amount of a test section's content a 
student must know for successful school performance and for 
admission. Two public educator panels reviewed every test item using 
two questions: (1) whether education students had the opportunity to 
learn this content; and (2) whether this knowledge was necessary for 
successful teaching performance. The PPST includes both multiple 
choice items (reading, mathematics, writing, and essay items). (Data 
analyses and scoring techniques aie briefly described). (GDC) 
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Selection of Study Participants 
The strategy employed in the validation of the Pre-Professional Skills 
Tests (PPST) required that panels of teacher educators and public school 
educators review the PPST and make professional judgments about the content of 
the tests. Ir this study, p defined as groups of experts from teacher 

education institutions and local school districts performed the test review 
functions. A committee of teacher educators and public school educators 
focused on test validity and minimum teacher education program admission 
scores. 

Panel Member Selection 

The nomination of panel members was made in response to a letter of 
request from the Tennessee Commissioner of Education. The materials mailed to 
the chief academic administrators of all teacher education program urn ^.s and 
selected local school district superintendents in Tennessee included the letter 
of request, institutions for nominating panel members, an overview of panel 
functions, the number of panelists to be nominated, and nomination forms. The 
local school districts selected to receive requests for nomi.-. ions of 
panelists were all districts in the county of the meeting sites and the 
adjacent counties. This local school district selection procedure minimized 
travel time for the public edicator panelists while providing representation of 
diverse school districts in Tennessee. 

The request for nominations from teacher education institutions soecified 
that the nominees should be faculty members who taught professior^al education 
courses taken by all students in the institution's teacher priuration program. 
The number of nominees requested from each institution varied from 2 to 20. 
The relative productivity of the teacher education program unit was used as the 
basis for determining the number of nominees requested. ■ total of 136 teacher- 
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educators participated in the review of the PPST. 

Superintendents of local school districts with high schools were asked to 
nominate instructional personnel such as curriculum specialists and experience 
teachers who were highly knowledgeable about the district's mathematics r 
reading/writing program for college preparatory students, A range of 1 to 10 
nominees for each panel (mathematics and readi ng/wri ti ng) was requested from 
the local school district. The size of the school district's enrollment was 
the fd'^tor used in specifying the number of nominees. The number of public 
educators for the 57 local school districts involved in the study was 108 for 
the panel. 

S tandards Committee Selection 

A group of 15 professional educators served as the Standards Committee for 
the PPST study. The committee consisted of eight teacher educators and seven 
public school educators who were currently employed in Tennessee. The members 
of the committee were selected by personnel in the Tennessee State Department 
of Education and were contacted by the study staff to obtain their commitment 
to serve on the committee. 

Functions of Participants 
The functions perfonned in the review of the PPST by teacher educators and 
public educators were characterized as panel activities. One teacher educator 
panel and two public educator panels were utilized in the study. 

Panel Preparation 

In advance of the panel fneetlngSi thp n^^nelists were sent preliminary 
materials related to their tasks. The materials provided to the members of 
each panel included an overview of the tasks to be performed, directions to be 
followed in reviewing the PPST, and sample sections of the data collection 
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forms to be used. Two versions of these materials were prepared for teacher 
educators and public educators. Test content specifications tor tne 
appropriate tests of the PPST battery were also supplied to each panelist. 

Teacher educators were encouraged t review the nature and content of the 
teacher education programs it their respective institutions prior to the panel 
session. The scope of the review was defined to include all courses that must 
be taken by all students. Public educators were advised to gather additional 
information on the content of the instructional program for college preparatory 
students in their respective school districts if necessary to make informed 
judgements about the PPST. 

Panel Functions 

Each teacher educator panel member reviewed all tests and sections of the 
PPST battery to make judgments on program relevance and performance standards. 
First, the panelists examined each test item in order to respond to two 
questions: (1) "Does a student nepd to know the content of this item in order 
to perfonn successfully in your institution's teacher preparation program?" and 
(2) "Should a student be required to know the answer to the item in order to be 
admitted to a teacher preparation program in Tennessee?" The response options 
to these questions were "Yes" and "No." Second, the panelist made a summary 
judgment on the test or section regarding the amount of the content that a 
stud it must know to perform successfully in the teacher preparation program 
of the panel i^.'s institution. The choices for responses were "almost all", 
"most," " a small amount," and "almost none." Finally, the panelist recorded 
the total number of questions on the test or sect. on tnat a student should be 
required to answer correctly in order to be admitted to the teacher preparation 
P^^ogram at the panelist's institution. With respect to the Essay section of 
the Writing Test, tne panelist indicated whether or not a teacher education 



program applicant (1) needs to be able to write an acceptable essay in order to 
perform successfully in the teacher preparation pro' --am of the panelist's 
institution and (2) should be required to write an acceptable essay in order to 
be admitted to a teacher preparation program in Tennessee. The recomn..nded 
minimum score for the essay (scale values of l-t) was also recorded. 

Each public educator panelist reviewed the assigned test{s) of the PPST 
battery to make judt;ments on adequacy of preparation and job relevance. First, 
the panel member reviewed each test item in order to respond to two questions: 

(1) "Do students in the college preparatory program In your district have the 
opportunity to learn the content of this item?" and (2) "Does a student need to 
know the content of this item in order to perfonn successfully as a teacher in 
Tennessee?" For the first question, the respondent used a criterion of "at 
least 90% of the district's college preparatory students" to select one of 
three responses: "Yes", "No", and "*" (do not know). The resnonse choices for 
the second question were "Yes" and "No". Second, the panel member made two 
summary jugments on the test or section regarding the amount of the content 
that (i) the district's college preparatory students have the opportunity to 
learn and (2) successful Tennessee teachers must know. The response options 
for these two judgments were "almost" all," "niost," "a small amount/ and 
"almost none". After reviewing the Essay section of the Writing Test, che 
panel member indicated whether or not the district's college preparatory 
students (1) have the opportunity to learn how to write an acceptable essay and 

(2) Should know how to write an acceptable essay in order to perfonn 
successfully as a Tennessee teacher. 

In order for the panelists to perfonn their functions, a current edition 
of the PPST battery was supplied by ETS for review by the panel members 
assigned to each test. The correct response to the multiple-choice questions 



on all tests were marked on the copies recieved by the panelists. The members 
of the readiny/writing panel also recieved 12 essays that were written by 
examinees who had taken the Writing Test of the PPST. Each o' the points on 
the score range for the essay (1-6) was represented by two essa s. "Each panel 
member worked independently in conducting the review based on instructions 
given by the study staff as described in the review booklet. 

Data Col lecti on 

The panel meetings of data collection were held during a two-week period 
in October, 1984. Two two-hour sessions (1:00 p.m. - 3:00 p.m. and 4:00 p.m. - 
6:00 p.m.) were conducted aw six sites across the state. The sites and dates 
of the meetings were Nashville on October 22nd, Jackson on October 23rd, 
Memphis on October 24th, Chattanooga cn October 29th, Knoxville on October 
30th, and Johnson City on October 31st. In addition to the involvement of the 
study staff, an ETS representative participated in the data collection meetings 
to ensure that test security was maintained. 

Data Analysis Techniques 
Data were obtained on selected demographic variables for members of the 
panels and the Standards Committee. These variables were geographic area, 
gender, and ethnic group. The analyses of these data for the separate panels 
and the committee consisted ot preparing frequency and percentage 
dist. ibutions. The years of professional education experience in Tennessee as 
teacher educators or public school educators was reported for the members of 
the respective panels. These data were summarized by obtaining frequency and 
percentage distributions, means, and standard deviations for the groups of 
panel i sts. 

The content review of the multiple choice questions on the PPST battery 



was conducted at the item and test or section levels by the teacher educators 
and public educators. At the item level, the panelist responded to review 
questions by choosiny structure-' response options. The choices were "Yes" or 
"No" for all review questions e> ,ept for the public educator panel question on 
adequacy of preparation, which also included "Do Not Know" as a third option. 
At the test or section level, four structured response options were used in 
making judgments. 

At the multiple-choice test item level, the initial step in the data 
analyses was to compute the frequency and percentage distributions of the 
panelists' responses to each review question. Using the percentages of "Yes" 
responses at the item level, the next step was to compute the percentages of 
items in a test or section that obtained more than specified percentages of 
"Yes" responses. The percentages established for summarizing across a test or 
section were more than 50%. 60%, 70%, 80%, and 90%. At the test or section 
level, the frequency and percentage distributions were computed for the 
responses of the panelists to the questions pertaining to each multi pi e-:hoi ce 
test or section. 

The review of the Essay section of the Writing Test required that each 
group of panelists— teacher educators and public educators—respond to two 
questions designed especially for the respective groups. The response options 
for each question were "Yes" and "No." Frequency and percentage distributions 
were computed for the responses of the panelists to each question. 

Each teacher educator i ecommended a performance standard for each test or 
section of the PPST. The recommended score for the Mathematics Test, Reading 
Test, and Writing--Multiple Choice Test consisted of the number of items that 
should be answered correctly for admission to the respondent's teacher 
education program. On the Writing-Essay Test, ;he panelists recommended the 
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minimum raw score on the scoring scale. The means of the raw score recommended 
by the panelists were computed Vor each test or section. The composite score 
for the Writing Test was derived by combining the raw scores for t^e Multiple 
Choice and Essay sections usiny an algebraic formula prescribed by £TS. The 
raw score means of the tests were converted to scaled score equivaicots based 
on the norms of the PPST, 
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